In video signals, strong spatial and temporal correlations exist. When video frames are partitioned into blocks, strong “intra frame” spatial correlation often exists between neighboring block in the same frame. Likewise, strong “inter frame” temporal correlation exists between corresponding blocks on temporally adjacent or neighboring frames in a video sequence.
Exploiting such correlation is the key to many techniques for achieving high data compression ratios in video coding. For example, widely used video coding standards (e.g., MPEG-1, MPEG-2, MPEG-4, H.261, H.263, H.264) utilize motion compensated prediction (“inter prediction”), which is effective for exploiting temporal correlation. In motion compensated prediction, a block to be coded searches its corresponding block in the neighboring frames and the difference signals are coded. Each new standard uses better temporal prediction technology than its predecessor to obtain better coding performance. For instance, half-pel motion compensated prediction was used in MPEG-2 to improve coding performance. H.264/AVC uses various techniques, including quarter-pel motion compensated prediction, variable block size motion compensation, and multiple reference frames, which significantly improve the prediction effectiveness and thus the coding performance. There are ongoing efforts to further improve the effectiveness of temporal prediction.
On the other hand, in H.264/AVC “intra prediction” exploits the spatial correlation among neighboring blocks in the same frame. In intra prediction, pixels neighboring the current block being coded form a predictive block along a given direction and the difference signals between the current block to be coded and the neighboring, predictive block, are coded. Intra prediction remarkably improves the coding performance when inter prediction can be not used, e.g., in an intra frame context, or when inter prediction is not effective, as when there are fast moving objects depicted in the video sequence.
FIG. 1 shows conventional prediction in video coding. A current image block 100 is an original block of pixels in a video frame 108 being coded. With respect to the above-introduced video coding standards, although sophisticated conventional inter prediction 102 techniques and sophisticated conventional intra prediction 104 techniques are both available and significantly improve coding performance (e.g., in H.264), each predicted block 106 that tries to emulate the current original block 100 being coded in a video frame 108 can conventionally only be predicted from one kind of conventional prediction mode: either inter frame prediction 102 or intra frame prediction 104. No predicted block 106 can simultaneously exploit both temporal and spatial correlations with other blocks.